Some issues on posterior predictive model checks for item response theory models ∗

Assessing the validity of model assumptions is a crucial part of any parametric statistical analysis. The question of interest is whether a particular specified model appears to provide an adequate fit. For item response theory models the goodness-of-fit problem has not been completely solved. The possible number of response patterns is large for even moderately long assessments, leading to sparse contingency table so the standard chi-square tests do not apply directly. In the last years several solutions have been developed to address this problem. Some of them considered limited information statistics but there is no universally accepted model fit measure (Cagnone and Mignani, 2007; Maydeu-Olivares and Joe, 2008). Furthemore, a number of researchers also recommended the use of a parametric bootstrap (Mavridis et al., 2007). Another alternative is the Bayesian approach, which provides not only an optimal estimation of the parameters but also the associated uncertainty. One well-known procedure for Bayesian model fit is the posterior predictive model checking (PPMC). The method was firstly developed by Rubin (1984) and was later extended to include general discrepancies by Gelman et al. (1996). The method compares the observed data with replicated data generated or predicted by the model using a number of diagnostic measures that are sensitive to model misfit. Any systematic differences between aspects of the observed data set and those of the replicated data sets indicate a failure of the model to explain those aspects of the data. The choice of discrepancy measures is crucial in the application of the PPMC method. Discrepancy measures should take into account characteristics of the model relating to the scientific purposes to which the inference will be applied and to measure aspects of the data not directly addressed by the probability model. The results of the chosen discrepancy measure can be investigated by a graphical approach. Graphical display is the most natural and easily tool to implement posterior predictive checks. If graphical displays do not suffice or are cumbersome, one can use a tail-area probability, also known as a posterior predictive p-value (PPP-value). Wu et al. (2014) proposed the relative entropy posterior predictive model checking (RE-PPMC) approach to complement the original procedure so that an arbitrary assertion of graphical comparison can be avoided. PPMC is a powerful and flexible tool and has many advantageous properties. By constructing the reference distribution empirically, PPMC overcome critical problems of traditional techniques. Furthermore, PPMC incorporates uncertainty of model parameter estimates into ∗Presented at the second internal meeting of the FIRB (“Futuro in ricerca” 2012) project “Mixture and latent variable models for causal-inference and analysis of socio-economic data”, Rome (IT), January 23-24, 2015